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EXECUTIVE  SUMMARY 


OBJECTIVE 

This  rqxHt  compares  measured  torward-looking  infrared  (FUR)  system  detection  ranges  of 
a  target  ship,  the  Research  Vessel  Point  Sur,  with  predictions  from  the  Electro-Optical  Tactical 
Decision  Aid  (EOTDA),  version  3.0.  The  EOTDA  was  primarily  devel(q}ed  by  the  Air  Fmrce, 
with  only  minimal  attention  ^plied  to  the  complexities  of  the  marine  environment.  The  objec¬ 
tive  of  this  case  study  was  to  evaluate  the  infrared  EOTDA  performance  for  a  target  in  an  qpen 
ocean  background. 

RESULTS 

Surface  metetx'ological  and  navigation  data  we  *  -eorded  aboard  the  Research  Vessel  Point 
Sur.  Eight  FUR  missions  were  flown  making  57  u  icc  ange  measurements.  The  meteoro¬ 
logical  and  navigation  data  were  input  to  the  EOTDA.  ar  :^  the  prediction  ranges  were  tabulated 
and  compared  with  the  reported  FUR  data.  Results  showed  lepsonable  accuracy  during  the 
clear-weatho*  portion  of  the  tests,  but  the  EOTDA  grossly  overp~tdicted  detection  ranges  when  a 
stratus  cloud  ceiling  prevailed.  Attempts  to  determine  integrated  target  temperature  from  therm¬ 
istors  mounted  on  the  siuface  of  the  ship  structure  were  unsuccessful.  Since  calibrated  target  and 
background  temperatures  were  unavailable  at  the  times  of  the  FUR  detection  range  c  measure¬ 
ments,  it  was  not  possible  to  isolate  the  portion  of  the  EOTDA  most  responsible  for  the  ovsrpre- 
dictions.  However,  the  transmission  mo^l  (LOWTRAN  7)  is  well-accepted  in  the  scienpri': 
community,  and  the  target  model  (TCM2)  of  the  EOTDA  performed  well  in  another  case  study 
during  the  same  field  tests.  This  would  make  the  water  background  model  of  the  EOTDA  most 
suspect 

RECOMMENDATIONS 

The  cause  of  the  overpredictions  must  be  determined.  Further  investigation  of  the  water,  sky, 
and  cloud  radiance  models  of  the  EOTDA  is  recommended,  especially  in  the  marine  environ¬ 
ment  Future  validations  with  FLIR  systems  should  be  accompanied  with  simultaneous  measure¬ 
ments  with  a  calibrated  imaging  system. 
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INTRODUCTION 


This  rep>ort  presents  a  set  of  airborne  forward-looking  infrared  (FLIR)  detection  range  data  as 
a  case  study  of  the  Electro-Optical  Tactical  Decision  Aid  (EOTDA)  version  3.0.  The  EOTDA  is 
a  microcomputer  code  that  predicts  the  performance  of  electro-optical  systems  {q>plied  to  various 
targets  for  strike  warfare  missions.  The  EOTDA  is  currently  being  adapted  to  a  workstation  plat¬ 
form  fcMT  inclusion  into  the  Tactical  Environmental  Support  System  version  3.0  (TESS(3))  and 
the  Thctical  Aircraft  Mission  Planning  System  (TAM^)  version  6.0,  under  the  direction  of 
Naval  Research  Laboratory  (NRL),  Monterey,  California. 

The  EOTDA  supports  three  optical  regions,  the  long- wave  infrared  (8-12  pm),  visible 
(0.4-0.9  pm),  and  laser  (1.06  pm).  The  scope  of  this  report  is  limited  to  studying  the  infrared 
(8-12  m)  range  predictions  for  an  airborne  FUR  sensor  directed  toward  a  ship  target  against  an 
open  ocean  background. 

The  detection  ranges  presented  in  this  report  were  collected  during  field  experiments  con¬ 
ducted  from  29  July  to  4  August  1992  off  the  coast  of  Monterey,  California.  The  purpose  of  the 
experiment  was  to  collect  meteorological  and  ship  temperature  data,  infrared  (IR)  images,  and 
airborne  FLIR  detection  range  data  in  support  of  ongoing  research  at  the  Naval  Command,  Con¬ 
trol  and  Ocean  Surveillance  Center  (NCCOSC),  Research,  Development,  Test  and  Evaluation 
Division  (NRaD)  and  the  Naval  Postgraduate  School.  The  Research  Vessel  Point  Sur  was  the 
meteorological  measurement  platform.  Meteorological  data  collected  onboard  the  Point  Sur  were 
converted  to  Terminal  Aerodrome  Forecast  (TAF)  code  format  for  input  into  the  EOTDA.  The 
Point  Sur  was  also  used  as  the  target  for  the  FLIR  and  IR  imaging  systems. 

The  FUR  detection  ranges  were  provided  by  aircrews  who  flew  the  missions  in  support  of 
the  field  tests.  Eight  missions  were  flown,  four  at  night  and  four  during  mid-morning,  making  a 
total  of  57  passes  at  the  target.  The  results  from  these  overflights  provide  the  primary  foundation 
for  this  report. 


BACKGROUND 


EOTDA  OVERVIEW 

Strike  warfare  planning  and  vulnerability  assessment  relies  on  software-driven  tactical  deci¬ 
sion  aids  (TDAs).  The  EOTDA  Mark  in  (Freni,  et  al.,  1993)  was  developed  by  the  coordinated 
effort  of  several  contractors  and  agencies,  with  Hughes  STX  Corporation  bringing  together  the 
final  product  under  the  direction  of  the  Air  Force  Phillips  Laboratory.  The  EOTDA  predicts  the 
performance  of  a  variety  of  sensors  against  a  variety  of  user-defined  targets  and  backgrounds. 
The  sensors  include  long- wave  IR,  television,  laser,  and  night- vision  goggles  (NVGs).  The  Navy, 
under  the  direction  of  NRL  Monterey,  leveraged  upon  the  Air  Force  effort  by  adding  Navy  and 
Marine  Corps  sensors  and  targets  to  the  already  developed  software.  Sensors  were  added  for 
several  naval  air  platforms  (e.g.,  A-6E,  FA-18,  P-3C,  and  F-14).  Two  ship  taigets  (a  frigate  and  a 
gunboat)  have  already  been  added,  and  a  user-definable  generic  ship  model  developed  by 
Georgia  Technology  Research  Institute  (GTRI)  is  currently  being  implemented. 

IR  sensor  performance  is  greatly  influenced  by  environmental  conditions.  The  meteorologi¬ 
cal  input  parameters  drive  the  outcome  of  the  target,  background,  and  atmospheric  transmission 
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calcxilations.  UnfOTtunately,  compromises  were  made  in  the  environmental  and  background 
models  of  the  EOTDA.  These  compromises  were  necessary  to  afford  reasonable  computer  run¬ 
times  and  to  accommodate  the  intended  operating  platfmm.  which  was  a  personal  computer  with 
an  80286  microprocessw  running  under  DOS.  Therefore,  a  simplified  two-layer  version  of 
LOWTRAN  7  (Kneizys,  et  al.,  1988)  determines  the  atmospheric  transmittance,  and  the  EOTDA 
target/backgroimd  model  employs  the  intermediate  grade  of  TCM2  (Blakeslee  and  Rodriquez, 
1993)  instead  of  the  more  refined  research  grade  software. 

The  target  and  background  temperatures  in  the  EOTDA  are  determined  by  the  intermediate 
grade  TCM2  model  developed  by  Georgia  Technology  Research  Institute.  TCM2  is  a  first- 
principles  thermal  contrast  model.  It  treats  the  target  and  background  as  a  series  of  three- 
dimensional  isothermal  spatial  nodes  with  specific  material  properties.  After  a  steady-state 
initialization  process,  an  energy  balance  solution  using  the  laws  of  heat  transfer  are  applied  to  the 
thermal  network  to  determine  the  nodal  heat  transfers.  Finally,  the  radiance  values  are  calculated 
in  the  desired  wave  band  for  each  node.  The  radiance  values  of  the  node  facets  that  are  visible  in 
the  field-of-view  of  the  sensor  are  integrated  to  determine  the  overall  target  radiance.  The  inter¬ 
mediate  grade  TCVQ  uses  a  one-dimensional  solution  and  allows  fewer  nodes,  thus  limiting 
target  resolution  compared  with  the  research  grade  TCM2.  The  background  is  modeled  as  a 
single  homogeneous  node  resembling  an  isothermal  plate  surrounding  the  target,  and  the  sky  is 
considered  a  uniform  hemisphere  covering  the  target.  The  complexities  of  the  background  and 
sky,  such  as  clutter,  sea-surface  wave-slope  reflections,  and  diffuse  scattering  and  cloud  effects 
are  ultimately  combined  into  a  homogenized  background  solution.  The  difference  between  the 
uniform  background  radiance  and  the  integrated  taiget  radiance  determines  the  zero-range  ther¬ 
mal  contrast.  Whether  this  thermal  contrast  is  representative  of  the  real-world  scene  depends  on 
many  conditions,  such  as  scene  complexity  and  human  recognition,  and  cueing  factors. 

The  required  meteorological  inputs  are  entered  in  a  user-friendly  format.  The  user  enters  the 
surface  and  dewpoint  temperatures  graphically  in  1®  increments  for  the  times  of  interest.  From 
these  temperatures,  corresponding  relative  humidity  values  are  calculated.  Any  of  17  different 
aerosol  models  can  be  selected.  The  wind,  rain,  cloud,  visibility,  and  other  meteorological  inputs 
are  entered  as  TAF  code  format.  It  is  likely  that  future  implementations  of  the  EOTDA  will  auto¬ 
mate  the  meteorological  inputs.  Automation  will  decease  the  possibility  of  human  error  and  will 
restore  input  data  precision  which  was  sacrificed  for  ease  of  use. 

The  operational  input  data  are  also  entered  in  a  user-friendly  format.  The  sensor,  target,  and 
background  configurations  are  selected  from  a  data  input  screen.  There  are  six  backgrounds,  18 
specific  targets,  several  user-definable  generic  targets,  and  many  sensors  to  choose  from. 

The  run  times  range  from  a  few  seconds  to  several  minutes,  depending  on  the  type  of  com¬ 
puter  (e.g.,  80486  vs.  80286),  the  target  complexity,  and  the  number  of  output  times  selected. 
Besides  the  numeric  and  graphical  outputs,  a  visualization  display  is  available  that  portrays  the 
radiance  values  of  the  backgroimd  and  each  target  node  pictorially. 

FLEET  USE  OF  THE  EOTDA 

Infrared  technology  was  employed  extensively  in  the  Persian  Gulf  during  Operation  Desert 
Storm,  but  the  EOTDA  was  not  widely  used  (Havener  and  Funk,  1991).  When  fee  EOTDA  was 
used,  it  favorably  influenced  mission  plaruiing  dedsions.  The  desert  background  model  proved 
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inaccunte  early  in  the  war,  but  this  problem  was  corrected  with  a  software  fix.  While  some  fore¬ 
casts  were  found  to  be  incorrect,  the  overall  predictions  were  considered  adequate.  However, 
there  was  little  feedback  on  the  accuracy  of  the  predictions,  because  the  Desert  Storm  pilots  and 
planners  only  required  general  (good,  marginal,  or  poor)  frxecasts.  The  most  difficult  problem 
reported  by  meteorologists  operating  the  EOTDA  was  not  with  the  EOTDA  itself,  but  getting 
accurate  battlefield  weather  conditions  and  frxecasts  to  input  into  the  model. 

Havener  and  Funk  offer  suggestions  based  on  Desert  Storm  for  improving  the  utility  of  the 
EOTDA.  One  was  to  tailor  the  output  to  the  needs  of  the  user.  The  level  of  detail  required  from 
the  EOTDA  varied  with  the  type  of  mission.  For  example,  an  aircraft  looking  for  targets  of 
oppotunity  only  needs  general  information  on  the  IR  conditions  for  various  times  of  the  day.  For 
an  aircraft  striking  a  specific  target,  more  detailed  information  on  detectioi  and  lock-on  ranges 
would  help  determine  the  best  weiq>omy  and  strategies  to  minimize  vulnerability.  Version  3  of 
the  EOTDA  addresses  this  suggestion  by  allowing  the  user  to  specify  the  contents  of  the  output. 
Other  suggestions  included:  provide  more  training  and  awareness,  automate  the  inputs,  and  pro¬ 
vide  a  usability  to  correct  and  redefine  the  backgrovmds  and  a  capability  to  define  new  targets. 

The  EOTDA  is  more  sophisticated  and  contains  more  capabilities  than  the  UFLR  model 
(Computer  Sciences  Corporation,  1986)  residing  in  the  current  shipboard  Tactical  Environmental 
Support  System.  It  is  also  much  easier  to  use  and  has  wider  distribution  than  previous  versions. 
Transitioning  the  EOTDA  into  TESS(3)  will  further  increase  the  availability  to  the  fleet.  It  is 
reasonable  to  assume  that  the  fleet’s  use  of  the  EOTDA  will  increase  dramatically  in  the  coming 
years.  Therefore,  it  is  crucial  that  the  EOTDA  be  properly  evaluated  to  determine  the  validity  of 
its  output  and  to  identify  any  problem  areas. 

VALIDATION  OF  THE  EOTDA 

Ideally,  the  limitations  and  accuracies  of  a  TDA  should  be  quantified  so  that  the  user  can 
prqserly  apply  the  information  it  provides.  However,  the  complex  modeling  requirements  and 
the  numerous  possibilities  of  envirorunental  and  physical  parameters  that  define  an  optical  scene 
make  calibrating  a  TDA  a  formidable  task  for  even  a  single  sensor  and  target  The  best  that  can 
be  done  in  most  cases  is  to  verify  the  basic  physics  involved  for  each  subroutine  and  then  per¬ 
form  a  variety  of  validation  tests  under  whatever  limited  conditions  naturally  occur  during  the 
testing  period.  Much  of  the  physics  of  the  EOTDA  is  from  well-accepted  methodology,  but  little 
has  been  done  to  quantitatively  validate  the  overall  performance. 

Sensitivity  tests  have  been  performed  (e.g.,  Shapiro,  1989).  However,  sensitivity  tests  do  not 
validate  the  model.  Sensitivity  tests  vary  the  input  parameters  for  a  given  base  case  to  show  how 
much  each  parameter  affects  the  results  for  a  given  scenario.  For  example,  small  amounts  of  rain 
or  battlefield-induced  contaminants  were  shown  to  have  a  large  effect  on  detection  range 
(Keegan,  1990).  In  general,  clouds  were  shown  to  reduce  temperature  contrast  which  tends  to 
reduce  detection  range.  However,  Shapiro  showed  that  the  presence  of  clouds  can  actually 
improve  detection  range  in  some  cases  by  delaying  or  preventing  a  temperature  contrast  reversal 
of  the  target  against  the  background.  While  sensitivity  tests  are  important  for  understanding  the 
behavirH-  of  the  model,  they  do  not  indicate  absolute  accuracy,  and  they  only  apply  to  the  base 
case  conditions. 

Real-world  validation  tests  cannot  fully  determine  accuracy  and  variability  either.  However, 
it  is  important  to  measure  TDA  performance  fi'om  an  end-user  perspective.  A  TDA  that  perfcHms 
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well  in  thecvy  but  cannot  be  tnisted  in  battle  is  dangerous.  The  result  will  be  either  poor  mission 
planning  w  a  TDA  that  will  be  ignwed.  End-to-end  evaluations  typically  compare  reports  of 
detection  ranges  from  air  crews  with  the  EOTDA  predictions.  Tests  controlled  by  researchers  are 
costly  and  large  data  samples  over  a  wide  range  of  conditions  are  usually  unobtainable.  Also,  it 
is  difficult  to  quantify  the  performance  differences  among  different  air  crews  and  different  air¬ 
craft.  In  spite  of  the  difficulties,  this  type  of  evaluation  is  crucial  in  deffning  the  usefulness  and 
limitations  of  the  TDA  in  real-world  situations,  and  often  identifies  problems  that  go  unnoticed 
in  modular  and  theoretical  testing. 

NRL  Monterey  has  conducted  several  evaluations  in  the  past  using  measurements  from  air¬ 
craft  out  of  naval  air  stations  at  Whidbey  Island,  Moffett  Field,  and  Lemoore.  Data  from  these 
measurements  have  led  to  TDA  improvements  and  were  used  to  compare  various  TDAs.  These 
comparisons  were  instrumental  in  deciding  to  include  the  EOTDA  Mark  in  into  TESS(3).  The 
aircraft  measurements  presented  in  this  report  advance  the  validation  process  and  also  illuminate 
some  concerns  that  need  to  be  investigated. 

The  primary  data  for  this  report  stem  from  an  IR  experiment  conducted  29  July  to  4  August 
1992  off  the  coast  of  Monterey,  California,  by  NRaD  in  cooperation  with  the  Naval  Postgraduate 
School.  The  purpose  of  the  experiment  was  to  collect  a  comprehensive  set  of  meteorological  and 
ship  temperature  data  for  validation  of  the  Performance  and  Range  of  Electro-Optical  Systems 
(P^OS)  detection  range  algorithm  (McGrath,  1992)  and  the  SHIPSIG  thermal  model  for  com¬ 
batant  sh4}S  (Ostrowski  and  Wilson,  1985;  Ostrowski,  1993).  The  initial  results  of  the  PREOS 
validation  were  documented  in  a  thesis  for  the  Naval  Postgraduate  School  (Kreitz,  1992).  The 
SHIPSIG  evaluation  was  combined  with  a  comparison  of  TCM2  and  presented  in  another  NRaD 
report  (McGrath,  Jensen,  and  Ostrowski,  1994).  The  field  data  are  further  utilized  in  this  report 
to  evaluate  the  EOTDA  Mark  HI. 


MEASUREMENTS 


OVERVIEW  OF  FIELD  TESTS 

A  field  experiment  was  conducted  29  July  to  4  August  1992  using  the  Research  Vessel  Point 
Sur  as  the  primary  measxirement  platform.  Point  Sur  is  a  135-foot  ship  that  is  owned  by  the 
National  Science  Foundation.  It  served  as  a  platform  for  meteorological  and  surface 
measurements  and  as  an  IR  target  for  the  airborne  FLIR  and  imaging  systems.  Scientists  from 
the  Naval  Postgraduate  School,  NRaD,  and  Naval  Surface  Warfare  Center  (NSWC)  were  aboard 
the  Point  Sur  to  conduct  the  onboard  measurements.  Figure  1  shows  the  ship  track  for  the  5-day 
cruise  as  recorded  by  the  Global  Positioning  System  (GPS)  receiver. 

Air  temperatures  ranged  from  11.5  to  17.8°C  during  the  cruise.  Sea  temperatures  were  from 
11.0  to  18.1®C,  with  a  warmer  sea  than  air  temperature  most  of  the  time.  Wind  speeds  ranged 
from  3  to  26  knots,  and  were  predominately  from  the  Northwest.  Stratus  clouds  dominated  until 
the  last  day  of  the  test  when  clear  skies  prevailed.  The  stratus  base  ranged  from  500  to  1300  feet 
according  to  the  aircrew  reports.  Cloud  tops  as  determined  from  radiosonde  relative  humidity 
gradients  ranged  from  1200  to  2100  feet. 
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Figure  1.  Research  Vessel  Point  Sur  GPS  track. 


SHIPBOARD  MEASUREMENTS 

Ground-truth  measurements  of  the  skin  temperatures  of  the  ship  were  collected  from  15 
thermistma  attached  to  the  ship,  and  a  hand-held  radiometer.  The  thermistors  were  mounted  on 
the  large-area  surfaces  of  the  ship  and  were  continuously  receded.  Spot  measurements  of  IR 
skin  temperature  were  made  several  times  daily  with  a  hand-held  radiometer  that  qierated  in  the 
8-12  |jun  band. 

Meteorological  data  were  collected  by  the  Naval  Pos^aduate  School  from  a  variety  of  ship¬ 
board  sensors.  The  surface  parameters  measured  included:  air  temperature,  sea  temperature, 
dewpoint  temperature,  wind  speed,  wind  direction,  relative  humidity,  pressure,  and  solar 
irradiance.  Vertical  profiles  of  meteorological  parameters  were  obtained  from  a  VAIS  ALA  RS80 
rawinsonde  system.  The  calibrated  accuracies  of  this  system  are  within  0.5  mb  pressure,  0.2°C 
temperature,  2%  relative  humidity,  and  2%  altitude.  Atmospheric  radon  concentrations  were  also 
recorded  as  an  aid  to  identifying  continental  versus  marine  air  masses.  Among  the  ship  naviga¬ 
tion  data  recorded  were  latitude,  longitude,  gyro  heading,  and  ship  speed. 

AIRBORNE  PLATFORM  MEASUREMENTS 

The  NRaD  airbcuce  platfcHm  is  a  Piper  Navajo  aircraft  that  is  outfrtted  with  navigation, 
metewological,  aerosol  and  thermal  image  recording  systems.  The  aircraft  flew  11  missions. 
MetecHTological  and  aerosol  data  were  collected  as  a  function  of  altitude  in  13  spiral  ascents,  each 
beginning  at  200  feet  and  climbing  to  5000  feet.  A  total  of  2603  image  frames  of  the  Point  Sur 
were  collected  with  the  AGA-780  Radiometric  Thermal  Imaging  System,  which  was  operated  in 
the  8-12  pm  band.  Additional  meteorological  and  aerosol  data  were  collected  at  200-foot  and 
500-foot  altitudes  during  the  image  collections.  Meteorolo^cal  and  navigational  parameto^ 
were  receded  at  5-second  intervals.  These  included  date,  time,  GPS  latitude  and  longitude, 
altitude,  air  temperature,  dewpoint  temperature,  relative  humidity,  and  atmospheric  pressure. 
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AIRBORNE  FLIR  MEASUREMENTS 

Aircraft  employing  an  8-12  pm  FLIR  system  widi  a  mercury-cadmium-telluride  (HgCdTe) 
sensor  detects  flew  range  detection  missions  during  the  field  tests.  Eight  missions  were  flown, 
for  a  total  of  57  FUR  target  runs  over  a  S-day  period.  Two  flights  were  scheduled  per  day,  one  at 
night  about  0600  hours  .jMT  (2300  PDT,  22(X)  PST)  and  the  other  in  the  morning  at  about  17(X) 
hours  GMT  ( 1(XX)  I  i^T.  0900  PST).  Exact  flight  times  are  listed  in  the  tables  of  Appendix  A. 

A  representative  from  the  Naval  Postgraduate  School  instructed  flight  crews  in  a  preflight 
briefing  before  each  mission.  The  aircrews  had  voice  and  visual  contact  with  the  Point  Sur 
h  jfore  making  detection  runs.  Runs  were  made  from  two  altitudes:  S(X)  and  1(XX)  feet.  For  con¬ 
sistency,  all  detection  ranges  were  determined  using  the  wide  (15  x  20  degree)  field  of  view  lens. 
Target  classification  and  identification  ranges  were  determined  using  the  narrow  (5  x  6.7  degree) 
field  of  view  lens.  Detection  was  defined  as  when  the  operator  first  notices  a  dot  on  the  display. 
The  aircrew  operator  recorded  the  detection  ranges  and  requested  supplemental  information  on 
data  sheets  provided  for  the  tests.  Display  images  were  preserved  on  video  Uq)e,  which  is  now 
archived  at  the  Naval  Postgraduate  School. 

EOTDA  OPERATION 

The  input  values  required  to  operate  the  EOTDA  include  meteorology,  navigation,  and 
operational  parameters.  Entering  the  operational  parameters,  such  as  type  of  sensor,  aircraft 
altitude,  and  ship  heading,  is  a  straightforward  transfer  of  measured  values.  In  contrast,  the 
meteorological  parameters  are  more  difricult  to  input  because  they  require  some  interpretation. 
Visibility,  for  example,  significantly  affects  det^tion  range  calculations,  but  is  difficult  to 
measure  accurately.  Wind  direction  and  speed  were  measured  continuously,  but  the  precise 
values  must  be  translated  to  TAF  code  to  operate  the  EOTDA,  which  tends  to  degrade  the 
precision. 

Comparison  tests  at  NRL  Monterey  noted  discrepancies  in  the  output  between  versions  2.0 
and  3.0.  When  the  metecxological  input  format  was  changed  from  the  tabular  format  in  version 
2.0  to  TAF  code  format  in  version  3.0,  some  software  errors  resulted.  It  is  possible  that  some  of 
the  detection  range  error  in  this  data  set  could  be  from  program  errexs  rather  than  model  perfor¬ 
mance.  However,  this  is  unlikely.  The  TAF  code  input  was  checked  for  proper  format,  and  the 
binary  input  files  created  by  the  EOTDA  were  spot  checked  to  verify  that  the  cloud  and  other 
meteorological  data  were  actually  getting  into  the  program.  Furthermore,  the  input  values  were 
from  carefully  monitored  in-situ  measurements,  rather  than  imprecise  forecasts.  It  is  therefexe 
reasonable  to  assume  that  the  output  results  are  primarily  from  model  performance. 

RESULTS 

DETECTION  RANGES 

Figure  2  is  a  scatter  plot  of  the  detection  ranges  predicted  by  the  EOTDA  compared  with  the 
detection  ranges  observed  by  the  aircrews.  The  same  data  are  presented  in  tabular  and  bar  graph 
form  in  Appendix  A  for  each  of  the  eight  missions.  The  data  show  a  tendency  of  the  EOTDA  to 
overpredict  the  detection  ranges.  There  was  no  significant  difference  between  the  results  of  the 
mtxning  flights  compared  with  the  night.  An  overcast  stratus  layer  prevailed  throughout  the 
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testing  poiod,  except  on  4  August,  the  final  night  of  the  field  tests.  The  EOTDA  predictions 
agree  more  closely  with  the  observed  detection  ranges  on  the  clear  weather  day,  4  August,  but 
grossly  overpredict  the  detection  range  dviring  the  stratus  cloud  conditions.  At  first  glance,  the 
EOTDA  cloud  model  appears  to  need  some  reEnements,  but  the  problem  is  not  that  simple.  A 
multitude  of  interrelated  factcnrs  is  involved.  Sensitivity  tests  (Shapiro,  1989)  show  that  clouds 
can  greatly  affect  detection  range,  but  the  effects  ate  maximum  during  the  daylight  hours  and 
minimal  at  night.  The  field  data  show  the  error  difference  between  the  clear  day  and  stratus  days 
is  large  regardless  of  whether  the  overflight  was  night  or  morning.  This  suggests  that  there  is 
more  than  simply  cloud  effects  causing  the  overprediction  of  ranges. 


0  s  10  IS  io  25 


OBSERVED  RANGE  (km) 

Figure  2.  Observed  detection  ranges  compared  with 
EOTDA  MDT  range  predictions. 


BACKGROUND  TEMPERATUF'^S 

The  tables  in  Appendix  B  tabulate  the  differences  between  the  TDA-calculated  background 
temperatures  and  the  measured  sea  temperatures.  Figures  3  and  4  use  the  data  from  these  tables 
to  compare  the  measured  and  calculated  water  background  temperatures  for  each  overflight. 
Figure  3  is  a  scatter  plot  of  the  temperature  values.  Figure  4  shows  the  magnitudes  of  the  temper¬ 
ature  differences  for  each  set  of  flights  (with  zero  being  perfect  agreement).  The  calculated 
values  of  the  physical  sea  temperature  are  taken  directly  from  the  EOTDA  ou^ut  file 
FACET.TMP.  The  measured  sea  temperatures  were  taken  aboard  the  Point  Sur  during  the  cruise 
from  a  thermistor  submerged  in  the  water.  The  thermistor  measurement  depth  ranged  from  about 
3  to  12  inches,  depending  on  the  ship  speed.  Radiometric  sea  temperatures  were  also  measured 
from  onboard  the  Point  Sur.  The  radiometric  values  were  taken  with  a  hand-held  8-12  |jun 
Everest  radiometer.  Figure  5  shows  that  the  onboard  thermistor  and  radiometric  temperatures  are 
in  close  agreement.  The  onboard  radiometric  and  thermistor  sea  temperatures  ranged  from  11.0 
to  18.1®C  over  the  entire  test  period,  with  an  average  agreement  of  0.1®C  and  standard  deviation 
of  0.3. 

Figure  4  shows  that  the  TDA  calculations  of  the  physical  sea  temperatures  are  within  ±  2.0‘’C 
on  the  stratus  weather  days.  On  the  clear  sky  day,  the  TDA  overpredicted  the  physical  sea  tem- 
peraUire  an  average  of  3.4*’C.  While  the  background  temperature  predictions  showed  the  largest 
errors  with  the  clear  weather  data,  the  range  predictions  were  most  accurate  for  this  same  data 
set.  Part  of  the  reason  for  this  apparent  contradiction  is  that  the  contribution  of  water  emissivity 
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to  total  back.^ound  radiance  is  minimized  at  large  angles  of  incidence,  as  shown  by  Fresnel’s 
equation  plotted  in  tigure  6  (Shq)iro,  1987).  The  incidence  angles  for  this  data  set  ranged  from 
85.8  to  89.6  degrees,  assuming  a  smooth  ocean  surface  and  isotropic  radiation.  Although  the 
real-wwld  sea  surface  is  never  perfectly  smooth  and  some  diffuse  radiation  is  always  present,  the 
angles  are  such  that  reflected  sky  radiance  makes  a  majcn:  backgroimd  contribution. 

The  measurements  indicate  that  the  most  significant  errors  in  the  background  temperature 
predictions  probably  stem  from  the  integrated  sky  radiance  and  ocean  surface  reflectance 
calculations.  Unfortunately,  validation  of  the  sky  contribution  is  not  possible  for  this  data  set. 

The  data  did  not  include  sky  radiance  measurements  or  simultaneous  radiometric  imaging  of  the 
target  and  background. 


11  12  13  14  15  16  17 

MEASURED  PHYSICAL  SEATEMPERATURE  (°C) 


Figure  3.  EOTDA  calculated  sea  temperature  compared  with  measured  values. 


TEST  PERIOD  (0549  30  JULTO  0532  04 AUG) 


Figure  4.  Background  temperature  differences  over  the  test  period  (57  overflights); 
Point  Sur  thermistor  values  minus  the  EOTDA  predicted  values. 
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Figure  5.  Comparison  of  radiometric  and  physical  sea  temperatures. 


Figiu-e  6.  Emissivity  and  reflectance  of  sea  water  as  a  function  of  incidence  angle. 


TARGET  TEMPERATURES 

The  target  temperatures  predicted  by  the  EOTDA  are  believed  to  be  reasonably  accurate  for 
the  data  set  presented  here.  Calibrated  thermal  images  of  the  target  were  analyzed  and  the  path 
effects  were  removed  (Hughes  and  McGrath,  1993).  These  corrected  values  were  compared  to 
the  Georgia  Technology  Research  Institute  program  TCM2  and  the  NSWC  program  SHIPSIG 
(McGrath,  Jensen,  and  Ostrowski,  1994).  The  model  and  target  temperatures  agreed  within 
±  2.0®C.  Table  1  shows  a  set  of  the  corrected  AGA-780  thermal  imager  temperatures  and  the 
TDA  target  temperatures  for  the  starboard  and  port  views  of  the  target.  The  overall  average  tem¬ 
perature  of  the  TDA  was  15.3®C,  and  the  AGA-780  image  temperature  average  was  17.5°C  . 

This  temperature  difference  is  easily  explained  by  time  difference  between  the  airborne  AGA 
measurements  and  the  aircraft  FLIR  measurements.  All  of  the  AGA  measurements  were  made  in 
the  afternoon,  when  a  hotter  target  is  likely.  The  TDA  validation  measurements  using  the  FUR 
system  were  made  late  at  night  and  at  mid-morning  when  a  cooler  target  temperature  is  expected. 
Thus,  the  2.2  degree  average  temperature  difference  is  not  unreasonable.  No  simultaneous  mea¬ 
surements  were  taken  with  operational  FUR  and  the  AGA  imager,  so  a  direct  correlation  of 
measured  and  predicted  temperatures  is  not  possible.  Attempts  to  correlate  the  FUR  and  AGA 
with  thermistor  recordings  of  the  target  also  proved  imsuccessfiil.  Twelve  thermistors  were 
attached  to  the  target  during  the  Eeld  experiment,  but  none  of  the  individual  thermistors 


9 


adequately  tracked  the  average  laiget  temperatures.  No  scheme  of  combining  the  thermistor 
values  has  yet  been  devised  that  represents  the  composite  shq)  temperature.  Until  a  better  data  set 
is  available,  the  target  model  cannot  be  prc^ly  validated.  Meanwhile,  it  appears  that  the  target 
model  is  functioning  with  an  accuracy  of  ±2.0'*C. 


Table  1.  Comparison  of  target  temperatures  from  the  AGA-780  imager  and  the  TDA 
predictions  fw  the  starboard  and  port  sides. 


CONCLUSIONS 


Since  the  EOTDA  program  development  was  primarily  directed  toward  Air  Force  applica¬ 
tions,  little  has  been  done  to  test  and  validate  the  marine  environment.  The  results  presented  here 
clearly  show  the  need  to  further  investigate  and  improve  the  EOTDA.  The  TDA  performed  well 
on  the  clear  sky  day  of  the  tests  and  performed  po<»’ly  on  the  stratus  days.  Tests  should  especially 
focus  on  die  transmission,  sky  radiance,  cloud,  and  water  background  models. 

It  is  hoped  the  EOTDA  c^ieration  can  be  fully  automated  in  the  future.  By  interfacing  the 
EOTDA  to  a  weather  fwecasting  system,  input  reliability  and  consistency  are  attainable,  and  the 
program  will  be  easier  to  use.  It  (qipears,  however,  that  the  TDA  predictions  are  unreliable  in 
some  situations.  Therefore,  an  experienced  human  operatm'  is  still  required  to  interpret  the 
results  and  adjust  die  predictions  based  on  feedback  from  similar  scenarios.  Fm*  example,  an 
qierator  who  knows  that  the  model  overpredicts  detection  ranges  in  an  open  ocean  background 
when  there  is  a  prevailing  stratus  cloud  ceiling  could  introduce  a  ccxTection  factor  (e.g.,  divide 
by  2.5)  that  would  make  the  results  more  accurate.  Hopefully,  furdier  validation  and  testing  will 
provide  solutions  that  do  not  require  intervention. 

The  current  EOTDA  is  a  vast  improvement  over  the  UFLR  program  that  is  cunendy  in 
TESS.  The  upgraded  user-interface  makes  the  program  much  easier  to  use,  and  the  mo^ls  push 
the  limit  of  the  original  intended  platfcxm,  a  PC  with  a  80286  processor.  The  fleet  use  of  the 
TDA  should  greatly  increase  now  that  the  program  is  easier  to  use  and  has  a  wider  distribution. 
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Ihciusion  into  TBSS(3}  should  further  increase  the  usage  of  die  EOTDA.  It  is  therefore,  critical 
that  die  model  be  more  fuUy  validated  and  any  problems  be  annotated  and  cmrected  as  quickly 
as  possible. 
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Appendix  A 

TABLES  OF  OBSERVED  DETECTION  RANGES  AND  EOTDA 

PREDICTIONS 
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Appendix  B 

TABLES  OF  MEASURED  SEA  TEMPERATURES 
AND  EOTDA  PREDICTIONS 
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Appendix  C 

BAR  CHARTS  OF  OBSERVED  AND  PREDICTED  DETECTION  RANGES 
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